OpenStack Compute (nova)

test_attach_cloned_encrypted_volume fails in nova-ceph-multistore job permanently

Bug #1912607 reported by Balazs Gibizer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Critical
Lee Yarwood

Bug Description

The job is red since 2021.01.20 20:18 UTC[1]. The failure is:

tempest.exceptions.BuildErrorException: Server 79ce2743-1b98-4867-8396-5c68c25f2cf3 failed to build and is in ERROR status
Details: {'code': 500, 'created': '2021-01-20T22:28:59Z', 'message': 'Build of instance 79ce2743-1b98-4867-8396-5c68c25f2cf3 aborted: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format'}

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [None req-3a8ebd8e-b22a-446f-ad35-16b12e7a1182 tempest-TestEncryptedCinderVolumes-88193216 tempest-TestEncryptedCinderVolumes-88193216] [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] Instance failed to spawn: nova.exception.ImageUnacceptable: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] Traceback (most recent call last):

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/compute/manager.py", line 2615, in _build_resources

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] yield resources

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/compute/manager.py", line 2384, in _build_and_run_instance

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] self.driver.spawn(context, instance, image_meta,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4061, in spawn

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] created_instance_dir, created_disks = self._create_image(

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4457, in _create_image

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] created_disks = self._create_and_inject_local_root(

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4585, in _create_and_inject_local_root

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] self._try_fetch_image_cache(backend, fetch_func, context,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 10164, in _try_fetch_image_cache

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] image.cache(fetch_func=fetch_func,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 273, in cache

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] self.create_image(fetch_func_sync, base, size,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 939, in create_image

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] prepare_template(target=base, *args, **kwargs)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py", line 360, in inner

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] return f(*args, **kwargs)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 270, in fetch_func_sync

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] fetch_func(target=target, *args, **kwargs)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4572, in clone_fallback_to_fetch

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] LOG.warning(

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 220, in __exit__

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] self.force_reraise()

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] six.reraise(self.type_, self.value, self.tb)

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/usr/local/lib/python3.8/dist-packages/six.py", line 703, in reraise

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] raise value

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 4564, in clone_fallback_to_fetch

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] backend.clone(context, disk_images['image_id'])

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] File "/opt/stack/nova/nova/virt/libvirt/imagebackend.py", line 1039, in clone

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] raise exception.ImageUnacceptable(image_id=image_id_or_uri,

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3] nova.exception.ImageUnacceptable: Image 0d4ef007-9ffd-4b8f-9bd2-1d9e3fca5f1b is unacceptable: Image is not raw format

Jan 20 22:28:53.073378 ubuntu-focal-rax-iad-0022637570 nova-compute[94729]: ERROR nova.compute.manager [instance: 79ce2743-1b98-4867-8396-5c68c25f2cf3]

Logstash signature: http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22is%20unacceptable%3A%20Image%20is%20not%20raw%20format'%5C%22

[1] https://zuul.opendev.org/t/openstack/builds?job_name=nova-ceph-multistore&project=openstack/nova

Tags: gate-failure
Balazs Gibizer (balazs-gibizer)
Changed in nova:
importance: Undecided → Critical
tags: added: gate-failure
Revision history for this message
Balazs Gibizer (balazs-gibizer) wrote :

example failure https://zuul.opendev.org/t/openstack/build/0865be3ce56746d199f7bef3c38663af/logs

Revision history for this message
Lee Yarwood (lyarwood) wrote :

61427 Jan 21 08:10:57.185061 ubuntu-focal-ovh-gra1-0022643495 nova-compute[94297]: WARNING nova.virt.libvirt.driver [None req-5443ffbb-803e-4e7e-86d0-e77a4eb650cf tempest-TestEncryptedCinderVolumes-1602703716 tempest-TestEncryptedCinderVolumes-1602703716] Image 6eb6b21b-cb90-47c7-81eb-6d2071c0f62c is not on my ceph and [workarounds]/never_download_image_if_on_rbd=True; refusing to fetch and upload.: nova.exception.ImageUnacceptable: Image 6eb6b21b-cb90-47c7-81eb-6d2071c0f62c is unacceptable: Image is not raw format

https://review.opendev.org/c/openstack/nova/+/771777 posted to skip it in this job

Balazs Gibizer (balazs-gibizer)
Changed in nova:
status: New → In Progress
assignee: nobody → Lee Yarwood (lyarwood)
Lee Yarwood (lyarwood)
Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 23.0.0.0rc1

This issue was fixed in the openstack/nova 23.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.